With the increasing down-scaling of integrated circuits and the increasingly demanding requirements to the speed of integrated circuits, transistors need to have higher drive currents with increasingly smaller dimensions. Fin Field-Effect Transistors (FinFET) were thus developed. The FinFETs include vertical semiconductor fins above a substrate. The semiconductor fins are used to form source and drain regions, and to form channel regions between the source and drain regions. Shallow Trench Isolation (STI) regions are formed to define the semiconductor fins. The FinFETs also include gate stacks, which are formed on the sidewalls and the top surfaces of the semiconductor fins. Although existing FinFET devices and methods of fabricating FinFET devices have been generally adequate for their intended purposes, they have not been entirely satisfactory in all respects. For example, a more flexible integration for forming fin and isolation structures is desired.